[auto] #29 automation: issue claim lease + auto-release on failed/aborted runs - #39
Closed
nutt-adam wants to merge 1 commit into
Closed
[auto] #29 automation: issue claim lease + auto-release on failed/aborted runs#39nutt-adam wants to merge 1 commit into
nutt-adam wants to merge 1 commit into
Conversation
… sweeper Implement claim leases so automation-claimed labels are never permanently stranded. Adds src/claim/ module with acquire, renew, release, and sweep operations backed by .tutti/state/claims/ persistence and gh CLI calls. Workflow executor auto-releases claims on failure. Scripts updated to write claim metadata and new release_claim.sh / sweep_stale_claims.sh added. Closes #29 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
📝 WalkthroughWalkthroughA comprehensive claim-lease system is introduced to manage GitHub issue automation claims. This includes a new Rust module implementing claim lifecycle management (acquire, renew, release, sweep), bash scripts for automated operations, GitHub Actions workflow integration, and auto-release logic triggered on workflow failures to prevent stale claims. Changes
Sequence DiagramsequenceDiagram
participant WF as GitHub<br/>Workflow
participant RustApp as Rust App<br/>(execute_workflow)
participant ClaimMgr as Claim<br/>Manager
participant GitHub as GitHub API<br/>(gh CLI)
participant FileSystem as File System<br/>(State)
WF->>RustApp: Run workflow with issue
RustApp->>ClaimMgr: acquire_claim(issue_num)
ClaimMgr->>FileSystem: save_claim(lease data)
ClaimMgr->>GitHub: add label (automation-claimed)
ClaimMgr->>GitHub: post audit comment
GitHub-->>ClaimMgr: success
Note over RustApp: Execute workflow steps
alt Workflow Succeeds
RustApp->>ClaimMgr: release_claim(issue_num, reason)
ClaimMgr->>GitHub: remove label
ClaimMgr->>FileSystem: delete claim file
ClaimMgr->>GitHub: post release comment
else Workflow Fails
RustApp->>ClaimMgr: release_claim(issue_num, "workflow failed")
ClaimMgr->>GitHub: remove label
ClaimMgr->>FileSystem: delete claim file
ClaimMgr->>GitHub: post release comment
end
Note over WF: Periodic sweep via cron/workflow
WF->>ClaimMgr: sweep_stale_claims()
ClaimMgr->>FileSystem: list all claims
loop For each claim
alt Claim expired
ClaimMgr->>GitHub: remove label
ClaimMgr->>FileSystem: delete claim file
ClaimMgr->>GitHub: post audit comment
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/sdlc-orchestrator.yml:
- Around line 78-83: The cleanup step is currently gated by "inputs.mode ==
'auto'" so smoke dispatches that run sdlc-smoke and call select_issue.sh never
get their claim released; remove the inputs.mode == 'auto' check from the step's
if expression (keep always() && (failure() || cancelled()) or change to always()
to run on success too) so the Release claim on failure/cancel step will run for
smoke runs, and add a safety check before invoking
scripts/automation/release_claim.sh to only call it if the claim file
(.tutti/state/auto/selected_issue.json) exists to avoid spurious errors;
reference the Release claim on failure/cancel step,
scripts/automation/release_claim.sh, and select_issue.sh/sdlc-smoke to locate
where to change the condition and add the existence check.
In `@docs/examples/tutti-codex-sdlc.toml`:
- Around line 233-238: The release_claim workflow step currently uses fail_mode
= "open", which lets errors in scripts/release_claim.sh leave the workflow green
and the issue still claimed; update the step with id "release_claim" to use
fail_mode = "closed" (make it blocking) so any GitHub/auth errors in
scripts/automation/release_claim.sh cause the workflow to fail and ensure the
claim release must succeed.
In `@scripts/automation/release_claim.sh`:
- Around line 24-41: The script currently ignores all failures from the gh issue
edit call and always removes the local claim file; change it so the local lease
(CLAIM_FILE) is deleted only when the GitHub label removal actually succeeded or
when the failure is the specific “label already absent” condition. Concretely,
run gh issue edit "$ISSUE_NUM" --repo "$REPO" and capture its exit status/output
instead of swallowing errors, check for success (exit code 0) or a recognized
“label not found/absent” message, and only then proceed to remove CLAIM_FILE
(.tutti/state/claims/${ISSUE_NUM}.json) and post the comment using RUN_ID;
otherwise log/emit the error and leave the claim file in place so the sweeper
can recover.
In `@scripts/automation/select_issue.sh`:
- Around line 73-75: The script currently writes leases to a claims dir derived
from OUT_FILE which can be overridden; change CLAIMS_DIR to the canonical
project-root path used by Rust and sweep scripts (the repository root +
/.tutti/state/claims) instead of "$(dirname "$OUT_FILE")/../../state/claims".
Update the CLAIMS_DIR assignment in select_issue.sh to compute the repo root
(e.g., via git rev-parse --show-toplevel or a reliable project-root heuristic)
and then mkdir -p that canonical "$REPO_ROOT/.tutti/state/claims" so
src/claim/mod.rs and scripts/automation/sweep_stale_claims.sh will always find
the lease.
In `@scripts/automation/sweep_stale_claims.sh`:
- Around line 48-61: The two subprocess.run(...) calls that call GitHub (the "gh
issue edit" and "gh issue comment") currently ignore failures yet the script
always calls os.remove(path) and increments released; change the logic in the
sweep loop so you only remove the claim file (os.remove(path)) and increment
released when both subprocess calls succeeded: either call subprocess.run with
check=True (or inspect .returncode == 0) for the "gh issue edit" and "gh issue
comment" invocations and skip deletion/increment on failure (log or capture the
error instead); ensure the change is applied where the current
subprocess.run(...) calls and the os.remove(path)/released += 1 statements
appear so a temporary GitHub failure does not delete the only recovery state.
In `@src/automation/mod.rs`:
- Around line 3066-3078: This auto-release runs for nested workflows; restrict
it to only the top-level owner run and let release_claim handle missing lease
files: inside execute_workflow_with_hooks, replace the current multi-check guard
(which uses claim::load_selected_issue_number and claim::load_claim) with a
single check that this invocation is the top-level owner run (e.g., an existing
is_top_level / owner_run_id equality or similar flag passed into
execute_workflow_with_hooks) and that
claim::load_selected_issue_number(project_root) returns Some(issue_num); then
call claim::release_claim(project_root, issue_num, &reason) directly (remove the
extra load_claim(...).is_some() guard) so the release_claim fallback logic still
runs when lease file is absent; keep the same error logging on Err.
In `@src/claim/mod.rs`:
- Around line 204-205: After calling add_claim_label(repo, issue_number) but
before returning on a failed save_claim(project_root, &lease) you must roll back
the GitHub label so the issue isn’t left stranded; modify the flow around
save_claim to, on Err(e), attempt to remove the label (e.g., call the inverse
function such as remove_claim_label(repo, issue_number) or remove_label) and
then return the original save error. Ensure removal failures are logged/ignored
(do not mask the original save_claim error) so
list_claims()/sweep_stale_claims() remain accurate.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 02308cf6-bffb-4b4a-8250-1e172c6fc1af
📒 Files selected for processing (8)
.github/workflows/sdlc-orchestrator.ymldocs/examples/tutti-codex-sdlc.tomlscripts/automation/release_claim.shscripts/automation/select_issue.shscripts/automation/sweep_stale_claims.shsrc/automation/mod.rssrc/claim/mod.rssrc/main.rs
Comment on lines
+78
to
+83
| - name: Release claim on failure/cancel | ||
| if: ${{ always() && inputs.mode == 'auto' && (failure() || cancelled()) }} | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| scripts/automation/release_claim.sh .tutti/state/auto/selected_issue.json "workflow ${{ job.status }}" |
There was a problem hiding this comment.
Smoke dispatches now have no claim cleanup path.
Line 62 still runs sdlc-smoke, and that workflow selects an issue via select_issue.sh. Because this cleanup step is gated to inputs.mode == 'auto', smoke runs can apply automation-claimed without any matching release on success or failure.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/sdlc-orchestrator.yml around lines 78 - 83, The cleanup
step is currently gated by "inputs.mode == 'auto'" so smoke dispatches that run
sdlc-smoke and call select_issue.sh never get their claim released; remove the
inputs.mode == 'auto' check from the step's if expression (keep always() &&
(failure() || cancelled()) or change to always() to run on success too) so the
Release claim on failure/cancel step will run for smoke runs, and add a safety
check before invoking scripts/automation/release_claim.sh to only call it if the
claim file (.tutti/state/auto/selected_issue.json) exists to avoid spurious
errors; reference the Release claim on failure/cancel step,
scripts/automation/release_claim.sh, and select_issue.sh/sdlc-smoke to locate
where to change the condition and add the existence check.
Comment on lines
+233
to
+238
| # 10) Release claim after successful completion | ||
| [[workflow.step]] | ||
| id = "release_claim" | ||
| type = "command" | ||
| run = "scripts/automation/release_claim.sh .tutti/state/auto/selected_issue.json 'workflow completed successfully'" | ||
| fail_mode = "open" |
There was a problem hiding this comment.
Make successful-run claim release blocking.
This is the only success-path cleanup in sdlc-auto. With fail_mode = "open", a GitHub/auth error in release_claim.sh still leaves the workflow green and the issue claimed, and the job-level cleanup only runs on failure/cancel.
💡 Minimal config fix
[[workflow.step]]
id = "release_claim"
type = "command"
run = "scripts/automation/release_claim.sh .tutti/state/auto/selected_issue.json 'workflow completed successfully'"
-fail_mode = "open"
+fail_mode = "closed"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # 10) Release claim after successful completion | |
| [[workflow.step]] | |
| id = "release_claim" | |
| type = "command" | |
| run = "scripts/automation/release_claim.sh .tutti/state/auto/selected_issue.json 'workflow completed successfully'" | |
| fail_mode = "open" | |
| # 10) Release claim after successful completion | |
| [[workflow.step]] | |
| id = "release_claim" | |
| type = "command" | |
| run = "scripts/automation/release_claim.sh .tutti/state/auto/selected_issue.json 'workflow completed successfully'" | |
| fail_mode = "closed" |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/examples/tutti-codex-sdlc.toml` around lines 233 - 238, The
release_claim workflow step currently uses fail_mode = "open", which lets errors
in scripts/release_claim.sh leave the workflow green and the issue still
claimed; update the step with id "release_claim" to use fail_mode = "closed"
(make it blocking) so any GitHub/auth errors in
scripts/automation/release_claim.sh cause the workflow to fail and ensure the
claim release must succeed.
Comment on lines
+24
to
+41
| # Remove the automation-claimed label (tolerate it already being absent). | ||
| gh issue edit "$ISSUE_NUM" --repo "$REPO" --remove-label "automation-claimed" 2>/dev/null || true | ||
|
|
||
| # Extract run_id from claim metadata if available. | ||
| RUN_ID=$(python3 -c " | ||
| import json,sys | ||
| d = json.load(open(sys.argv[1])) | ||
| print(d.get('claim', {}).get('run_id', 'unknown')) | ||
| " "$ISSUE_FILE" 2>/dev/null || echo "unknown") | ||
|
|
||
| # Post audit comment. | ||
| gh issue comment "$ISSUE_NUM" --repo "$REPO" \ | ||
| --body "🤖 **Claim released** — reason: ${REASON} (run \`${RUN_ID}\`)" \ | ||
| >/dev/null 2>&1 || true | ||
|
|
||
| # Remove claim state file. | ||
| CLAIM_FILE=".tutti/state/claims/${ISSUE_NUM}.json" | ||
| rm -f "$CLAIM_FILE" 2>/dev/null || true |
There was a problem hiding this comment.
Don't drop the local lease unless the GitHub label was actually removed.
This swallows every gh issue edit failure, not just “label already absent”, and then deletes .tutti/state/claims/${ISSUE_NUM}.json anyway. A transient GH/auth error will leave automation-claimed on the issue with no local lease left for the sweeper to recover.
💡 Suggested hardening
-# Remove the automation-claimed label (tolerate it already being absent).
-gh issue edit "$ISSUE_NUM" --repo "$REPO" --remove-label "automation-claimed" 2>/dev/null || true
+# Remove the automation-claimed label, but only tolerate "already absent".
+REMOVE_ERR=""
+if ! REMOVE_ERR=$(gh issue edit "$ISSUE_NUM" --repo "$REPO" --remove-label "automation-claimed" 2>&1); then
+ case "$REMOVE_ERR" in
+ *"not found"*|*"does not have"*)
+ ;;
+ *)
+ echo "Failed to remove automation-claimed from issue #${ISSUE_NUM}: $REMOVE_ERR" >&2
+ exit 1
+ ;;
+ esac
+fi
@@
-# Remove claim state file.
-CLAIM_FILE=".tutti/state/claims/${ISSUE_NUM}.json"
-rm -f "$CLAIM_FILE" 2>/dev/null || true
+# Remove claim state file only after label removal succeeded.
+CLAIM_FILE=".tutti/state/claims/${ISSUE_NUM}.json"
+rm -f "$CLAIM_FILE"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/automation/release_claim.sh` around lines 24 - 41, The script
currently ignores all failures from the gh issue edit call and always removes
the local claim file; change it so the local lease (CLAIM_FILE) is deleted only
when the GitHub label removal actually succeeded or when the failure is the
specific “label already absent” condition. Concretely, run gh issue edit
"$ISSUE_NUM" --repo "$REPO" and capture its exit status/output instead of
swallowing errors, check for success (exit code 0) or a recognized “label not
found/absent” message, and only then proceed to remove CLAIM_FILE
(.tutti/state/claims/${ISSUE_NUM}.json) and post the comment using RUN_ID;
otherwise log/emit the error and leave the claim file in place so the sweeper
can recover.
Comment on lines
+73
to
+75
| # Persist claim lease for the Rust-side sweeper / auto-release. | ||
| CLAIMS_DIR="$(dirname "$OUT_FILE")/../../state/claims" | ||
| mkdir -p "$CLAIMS_DIR" |
There was a problem hiding this comment.
Write the lease to the canonical claims directory.
src/claim/mod.rs and scripts/automation/sweep_stale_claims.sh always read .tutti/state/claims from the project root. If callers override OUT_FILE, $(dirname "$OUT_FILE")/../../state/claims can resolve somewhere else, and the Rust auto-release / stale sweeper won't find the lease you just created.
💡 Minimal fix
-CLAIMS_DIR="$(dirname "$OUT_FILE")/../../state/claims"
+CLAIMS_DIR=".tutti/state/claims"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/automation/select_issue.sh` around lines 73 - 75, The script
currently writes leases to a claims dir derived from OUT_FILE which can be
overridden; change CLAIMS_DIR to the canonical project-root path used by Rust
and sweep scripts (the repository root + /.tutti/state/claims) instead of
"$(dirname "$OUT_FILE")/../../state/claims". Update the CLAIMS_DIR assignment in
select_issue.sh to compute the repo root (e.g., via git rev-parse
--show-toplevel or a reliable project-root heuristic) and then mkdir -p that
canonical "$REPO_ROOT/.tutti/state/claims" so src/claim/mod.rs and
scripts/automation/sweep_stale_claims.sh will always find the lease.
Comment on lines
+48
to
+61
| # Remove label. | ||
| subprocess.run( | ||
| ["gh", "issue", "edit", str(issue_num), "--repo", repo, "--remove-label", "automation-claimed"], | ||
| capture_output=True, | ||
| ) | ||
| # Post audit comment. | ||
| subprocess.run( | ||
| ["gh", "issue", "comment", str(issue_num), "--repo", repo, | ||
| "--body", f"🤖 **Claim released** — reason: lease expired (sweeper, run `{run_id}`)"], | ||
| capture_output=True, | ||
| ) | ||
| # Remove claim file. | ||
| os.remove(path) | ||
| released += 1 |
There was a problem hiding this comment.
Only delete a stale lease after the GitHub release succeeds.
Both subprocess.run(...) calls ignore their return codes, but os.remove(path) and released += 1 still happen. A temporary GitHub failure will report the claim as released while leaving the issue labeled and deleting the only state the next sweep could use to repair it.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/automation/sweep_stale_claims.sh` around lines 48 - 61, The two
subprocess.run(...) calls that call GitHub (the "gh issue edit" and "gh issue
comment") currently ignore failures yet the script always calls os.remove(path)
and increments released; change the logic in the sweep loop so you only remove
the claim file (os.remove(path)) and increment released when both subprocess
calls succeeded: either call subprocess.run with check=True (or inspect
.returncode == 0) for the "gh issue edit" and "gh issue comment" invocations and
skip deletion/increment on failure (log or capture the error instead); ensure
the change is applied where the current subprocess.run(...) calls and the
os.remove(path)/released += 1 statements appear so a temporary GitHub failure
does not delete the only recovery state.
Comment on lines
+3066
to
+3078
| // Auto-release claim on workflow failure so issues don't stay permanently blocked. | ||
| if !result.success | ||
| && let Ok(Some(issue_num)) = claim::load_selected_issue_number(project_root) | ||
| && let Ok(Some(_lease)) = claim::load_claim(project_root, issue_num) | ||
| { | ||
| let reason = format!( | ||
| "workflow `{}` failed (steps {:?})", | ||
| result.workflow_name, result.failed_steps | ||
| ); | ||
| if let Err(e) = claim::release_claim(project_root, issue_num, &reason) { | ||
| eprintln!("claim: auto-release failed for issue #{}: {}", issue_num, e); | ||
| } | ||
| } |
There was a problem hiding this comment.
Scope claim auto-release to the top-level owner run.
execute_workflow_with_hooks() is also used for nested workflows and hooks (Line 1302, Line 2859, Line 2936). A nested workflow that returns success = false under an open parent fail mode (Line 1329) will hit this block and release the selected issue while the outer automation keeps running. The extra load_claim(...).is_some() guard also skips claim::release_claim()'s fallback when the issue file exists but the lease file was never written.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/automation/mod.rs` around lines 3066 - 3078, This auto-release runs for
nested workflows; restrict it to only the top-level owner run and let
release_claim handle missing lease files: inside execute_workflow_with_hooks,
replace the current multi-check guard (which uses
claim::load_selected_issue_number and claim::load_claim) with a single check
that this invocation is the top-level owner run (e.g., an existing is_top_level
/ owner_run_id equality or similar flag passed into execute_workflow_with_hooks)
and that claim::load_selected_issue_number(project_root) returns
Some(issue_num); then call claim::release_claim(project_root, issue_num,
&reason) directly (remove the extra load_claim(...).is_some() guard) so the
release_claim fallback logic still runs when lease file is absent; keep the same
error logging on Err.
Comment on lines
+204
to
+205
| add_claim_label(repo, issue_number)?; | ||
| save_claim(project_root, &lease)?; |
There was a problem hiding this comment.
Rollback the GitHub label if local claim persistence fails.
If add_claim_label() succeeds and save_claim() returns an error, the issue is left with automation-claimed but no lease file. That makes list_claims() / sweep_stale_claims() blind to the stranded claim.
💡 Suggested rollback
add_claim_label(repo, issue_number)?;
- save_claim(project_root, &lease)?;
+ if let Err(err) = save_claim(project_root, &lease) {
+ let _ = remove_claim_label(repo, issue_number);
+ return Err(err);
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| add_claim_label(repo, issue_number)?; | |
| save_claim(project_root, &lease)?; | |
| add_claim_label(repo, issue_number)?; | |
| if let Err(err) = save_claim(project_root, &lease) { | |
| let _ = remove_claim_label(repo, issue_number); | |
| return Err(err); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/claim/mod.rs` around lines 204 - 205, After calling add_claim_label(repo,
issue_number) but before returning on a failed save_claim(project_root, &lease)
you must roll back the GitHub label so the issue isn’t left stranded; modify the
flow around save_claim to, on Err(e), attempt to remove the label (e.g., call
the inverse function such as remove_claim_label(repo, issue_number) or
remove_label) and then return the original save error. Ensure removal failures
are logged/ignored (do not mask the original save_claim error) so
list_claims()/sweep_stale_claims() remain accurate.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated SDLC cycle for #29.
Summary by CodeRabbit
New Features
Chores